ASVspoof 2015: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plan

The ASVspoof initiative follows on from the first special session in spoofing and countermeasures for automatic speaker verification (ASV) held during the 2013 edition of INTERSPEECH in Lyon, France [1]. The vision behind that first edition was to promote the consideration of spoofing, to encourage the development of anti-spoofing countermeasures and to gather a community to design, collect and distribute standard databases with standard evaluation protocols and metrics. Such an initiative is deemed vital in order to address weaknesses in research methodologies common to the majority of previous work. In the past, both spoofing attacks and countermeasures have generally been developed with full knowledge of a particular ASV system used for vulnerability assessments [2]. Similarly, countermeasures have been developed with full knowledge of the spoofing attack which they are designed to detect. This is clearly unrepresentative of the real use case scenario in which neither the specific attack, much less the specific algorithm, can ever be known a priori. It is thus likely that the prior work has as much over-exaggerated the threat of spoofing as it has the performance of countermeasures. The ASVspoof challenge has been designed to address these shortcomings and to support, for the first time, independent assessments of vulnerabilities to spoofing and of countermeasure performance. While preventing as much as possible the inappropriate use of prior knowledge, the challenge aims to stimulate the development of generalised countermeasures with potential to detect varying and unforeseen spoofing attacks. The first evaluation, ASVspoof 2015, is being held within the scope of a special session at INTERSPEECH 2015 and with a focus on spoofing detection. Expertise in ASV is therefore not a prerequisite to participation in the 2015 edition. ASVspoof 2015 participants are invited to develop spoofing detection algorithms and to submit results for a freely available standard database and according to the standard protocol and metrics described in this evaluation plan.